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Al"  Ml  A  i 


The  present  Investigation  attempted  t  Isolate  the  most  critical  skills  and  procedures  within  each 
of  the  stages  comprising  replacement  air  group  (RAG)  training  In  the  F-4  aircraft.  For  each  of  the 
stages  analyzed,  a  small  set  of  graded  Items  were  selected  on  the  basis  that  they  could  adequately 
discriminate  among  replacement  pilots  categorized  as  "above  average,"  "average,"  or  "below 
average, "according  to  their  final  overall  RAG  grade.  Such  Items  were  found  to  be  highly  predic¬ 
tive  of  the  stage  grade  from  which  they  were  obtained.  A  multiple  R  of  ,839  was  obtained, 
predicting  the  final  RAG  grade  when  only  five  of  the  selected  maneuvers  were  entered  Into  the 
equation.  These  findings  Indicate  the  feasibility  of  Isolating  a  small  set  of  skills  and  procedures 
which  will  be  highly  predict!  v  e  of  pilot  performance  In  the  RAG.  It  Is  suggested  that  such 
"critical"  Items  should  form  the  basis  from  which  an  adequate  measure  of  fleet  performance  might 
be  developed, 
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SUMMARY  PAGE 


THE  PROBLEM 


A  persistent  problem  In  naval  aviation  has  been  the  lack  of  adequate  criteria 
for  pilot  performance  In  operational -type  fleet  aircraft.  Any  useful  measure  should 
Include  estimates  of  proficiency  In  basic  flying  skills  as  well  as  the  unique  skills  dic¬ 
tated  by  the  mission  objectives  for  a  particular  aircraft.  The  replacement  air  group 
(RAG)  seems  to  be  the  phase  of  training  wherein  these  skills  could  be  most  easily 
Isolated.  The  RAG  Is  the  postgraduate  phase  of  training  where  the  replacement  pilot 
Is  first  Introduced  to  thn  operations  of  a  fleet  aircraft.  Since  the  RAG  Is  the  final 
phase  of  preparation  fo;  fleet  operations,  It  seems  reasonable  that  the  most  Important 
skills  Identified  for  this  level  of  training  should  also  be  of  Importance  In  the  estimation 
of  fleet  performance.  The  present  Investigation  attempted  to  Isolate  the  most  critical 
skills  and  procedures  within  each  of  the  stages  comprising  RAG  training  for  the  F-4 
aircraft. 


FINDINGS 


For  each  of  the  stages  analyzed,  a  small  set  of  skills  and  procedures  were 
selected  on  the  basts  that  they  could  adequately  discriminate  among  replacement  pilots 
categorized  as  "above  average,**  "average,"  and  "below  average,"  according  to  their 
final  overall  RAG  grade.  Such  Items  were  also  found  to  be  highly  predictive  of  the 
overall  stage  grade  from  which  they  were  obtained.  A  multiple  correlation  coefficient 
(corrected  for  shrinkage)  of  .839  was  obtained  predicting  the  final  overall  RAG  grade 
when  only  five  of  the  selected  maneuvers  were  entered  Into  the  equation.  These 
findings  Indicate  that  skills  and  procedures  can  be  Isolated  which  are  highly  predictive 
of  performance  tn  the  RAG  based  on  the  final  grade  a  student  .  ecelves.  It  Is  suggested 
that  such  "critical"  skills  and  procedures  should  form  the  basis  from  which  an  adequate 
measure  of  fleet  performance  can  be  developed. 
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INTRODUCTION 


At  present  the  naval  aviation  training  flight  syllabus  oomlit*  of  four  phases  i 
primary/  baste,  advanced,  and  tha  raplaeamant  air  group  (RAG).  Tho  flnt  three  of 
thoio  aro  oontldorod  tho  undorgraduato  lovol  of  training.  Upon  eomplotlon  of  tho 
advaneod  phaio,  tho  itudont  pilot  ti  awarded  hit  wings  end  Is  designated  a  naval  avia¬ 
tor.  Tho  RAG  Is  tho  postgraduate  phase  of  training  wherein  the  student  Is  first  Intro¬ 
duced  to  the  operations  of  a  fleet-type  aircraft.  Upon  completion  of  this  phase/  he  Is 
assigned  to  a  fleet  squadron. 

To  date,  most  of  the  past  research  effort  has  been  devoted  to  the  development  of 
criteria  for  a  reliable  prediction  system  based  at  the  undergraduate  level  of  training. 
Most  often  the  pass/attrlto  dichotomy  or  the  final  overall  flight  grade  has  been  used. 

The  success  of  such  efforts  within  the  laboratory  Is  well  documented.  Nevertheless,  the 
fact  remains  that  a  pilot  Is  trained  to  become  an  Integral  part  of  an  operational  fleet 
squadron.  Consequently,  the  best  criterion  of  pilot  performance  must  necessarily  be  the 
manner  In  which  he  fulfills  the  mission  ob|ectlves  of  his  particular  aircraft.  In 
developing  performance  measures,  the  replacement  air  group  seems  to  be  a  fertile  area 
for  Investigation  because  Its  activities  are  so  similar  to  actual  fleet  operations.  It  Is 
highly  encouraging  to  note  that  several  recent  attempts  have  been  made  to  extend  the 
scope  of  research  efforts  Into  the  RAG.  Bale,  Rlckus,  and  Ambler  (1)  reported  that 
certain  grades  during  stages  of  undergraduate  training  were  significantly  related  to  RAG 
performance  defined  on  a  pass/fall  dichotomy.  Using  a  factor  analytic  procedure,  Bale, 
Smith,  and  Ambler  (2)  reported  substantial  commonality  among  skills  Identified  In  both 
the  undergraduate  and  postgraduate  phases  of  training. 

RAG  training,  like  the  undergraduate  phases,  Is  broken  down  Into  a  series  of 
stages.  The  number  of  stages  and  content  of  each  are  highly  dependent  upon  the  mission 
ob|ecttves  for  a  particular  aircraft.  Within  each  stage,  the  replacement  pilot  Is 
expected  to  demonstrate  proficiency  In  the  performance  of  a  wide  variety  of  skills  and 
procedures.  It  Is  from  the  ratings  which  he  receives  on  ea  h  of  these  Items  that  his 
stage  grades  are  determined.  It  seems  reasonable  that  within  each  stage,  certain  of 
these  rMlis  and  procedures  should  be  of  greater  Importance  than  others.  It  should  be 
possible  to  Isolate  a  small  set  of  "critical"  Items  that  would  discriminate  among 
replacement  pilots  of  differing  ability  levels.  Such  Items,  it  Is  contended,  should  form 
the  basis  upon  which  an  adequate  fleet  performance  measure  might  be  constructed.  The 
purpose  of  the  present  Investigation  was  to  Isolate  the  most  Impotent  skills  and 
procedures  within  each  stage  of  the  F-4  RAG  tn  an  attempt  to  Identify  those  elements 
which  would  then  form  the  basis  of  a  fleet  performance  criterion  for  the  F-4  aircraft. 
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PROCEDURE 


The  tamp  It  group  conilittd  of  52  first-tour  replacement  pilots  assigned  to  VF-101 , 
the  tail  eeait  squadron  for  RAO  training  In  the  P-4  aircraft  botwssn  Dtetmbtr  1969  and 
Junt  1971 .  At  that  tlmt  ths  training  syllabus  for  flnt-tour  replacement  pilots 
oomlitsd  of  ths  following  stages)  Familiarisation  (PF),  Instruments  (PI),  Weapons 
Systems  (PS),  Conventional  Wtaponi  (PW),  Taetlei  (PT),  and  Flsld  Mirror  Landing 
Practlce/Carrler  Qualification  (FMLP/CQ),  A  brief  description  of  each  of  these  stages 
Is  presented  In  the  Appendix.  Student  flight  records  were  obtained  and  Item  analyses 
performed  for  each  of  the  above  steges  except  PW .  This  stage  was  omitted  due  to  the 
Incompleteness  of  record*. 


With  the  final  RAG  grade  as  a  criterion,  the  sample  was  divided  Into  three 
approximately  equal  groups.  In  this  manner,  the  sample  replacement  pilots  were 
categorised  as  "above  average, "  "average, 11  or  "below  average. "  An  attempt  was  then 
made  to  select  Items  that  would  correctly  categorize  the  sample  Into  these  classifications, 
based  on  their  final  overall  grade. 

The  grading  system  for  the  east  coast  RAG  requires  the  Instructor  to  rate  his 
students'  performance  on  each  skill  and  procedure  as  falling  within  one  of  four 
categories)  Above  Average  (AA),  Average  (A),  Below  Average  (BA),  or  Unsatisfactory 
(U).  For  each  Item  within  a  stage,  the  number  of  AAs,  BAs,  and  Us  were  recorded  for 
each  student  across  all  hops.  Items  within  a  stage  were  selected  for  further  analysis 
which  could  reasonably  discriminate  among  the  sample  according  to  their  categorization 
as  "above  average, "  "average, "  or  "below  average. "  Ai  Item  was  considered 
disci Imlnatlng  If  the  "above  average"  students  received  a  relatively  largo  number  of  AA 
ratings  while  the  "below  average"  students  received  a  relatively  large  number  of  BA  and 
U  ratings. 

For  each  of  the  Items  selected  In  this  manner,  a  difference  score  was  obtained 
for  each  student, reflecting  the  distribution  of  AAs,  BAs,  and  Us.  For  each  student 
the  number  of  AAs,  BAs,  and  Us  were  summed  across  the  entire  stage  for  each  Item. 
These  category  sums  were  weighted  by  +1 ,  -1,  and  -2  ,  respectively,  and  then  summed  in 
order  to  yield  an  overall  difference  score.  The  resulting  Item  difference  scores  were 
then  standardized  and  transformed  to  ,T  scores.  These  derived  stores  for  each  of  the 
selected  Items  were  then  related  to  the  Individual  stage  grade  as  well  as  the  final  RAG 
grade  In  a  series  of  correlational  and  regression  analyses. 


RESULTS 


For  the  PF  stage,  i  of  13  graded  skills  and  procedures  were  selected.  These 
Included  Headwerk,  Basle  Alrwark,  Maneuvers,  VFR  Glide  Slope  Central,  and  Landings* 


l 


For  the  PI  stage,  non*  of  the  Items  wa#  found  to  adequately  dtierlmlnate  among  the 
three  categories  of  replacement  pilots.  Three  of  17  Items  were  selected  from  the  PS 
stage;  these  Included  Altitude  Control ,  Headword,  and  Baste  Alrwork,  For  the  PT 
stage,  5  of  15  were  selected,  Including  Aggressiveness,  Offensive  Air  Combat 
Maneuvering,  Lookout  Doctrine,  Headwork,  and  Basic  Alrwork.  Four  of  13  Items  were 
selected  from  the  FMLP/CQ  stage;  these  Included  Speed  Control,  Glide  Slope  Control, 
Scan,  and  Power/Nose  Control .  Correlations  were  computed  between  the  derived 
scores  for  each  of  these  17  Items  with  the  PAG  grade  for  each  stage  as  well  as  the  final 
overall  RAG  grade.  These  results  are  summarized  In  Table  I. 


Table  I 

Summary  of  Zero-Order  Correlations  Between 
Item  Scores  and  RAG  Grades 


Correlations  With: 


Item 

Stage 

Stage  Grade 

Total  Grade 

Headwork 

PF 

.510 

.035 

Basic  Alrvork 

PF 

.699 

.276 

Landings 

PF 

.655 

.194 

VFR  G/S  Control 

PF 

.653 

.244 

Maneuvers 

PF 

.459 

,108 

Altitude  Control 

PS 

.434 

.302 

Headwork 

PS 

.291 

.121 

Baste  Alrwork 

PS 

.255 

.255 

Aggressiveness 

PT 

.731 

.447 

Offensive  ACM 

PT 

.867 

,440 

Lookout  Doctrine 

PT 

.616 

.26  5 

Headwork 

PT 

.612 

.217 

Basic  Alrwork 

PT 

.639 

.410 

Speed  Control 

FMLP/CQ 

.579 

.611 

Glide  Slope  Control 

FMLP/CQ 

.645 

.549 

Scan 

FMLP/CQ 

.534 

.439 

Power/Nose  Control 

FMLP/CQ 

.69  5 

.646 

A  series  of  regression  analyses  was  then  performed  to  determine  the  relative 
contribution  of  each  of  the  Items  within  a  stage  to  the  overall  stage  grade.  By  a 
forward  selection  regression  procedure,  multiple  correlation  coefficients  (corrected 
for  shrinkage)  of  .016,  .495,  ,098,  and  .760  were  obtained  for  the  PP,  PS,  PT,  and 
FMLP/CQ  stages,  respectively .  All  of  the  obtained  R's  were  highly  significant , 
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A  regression  analyst!  was  than  performed  In  attempting  to  predict  the  final  overall 
RAG  grade  from  the  17  Items  selected.  The  obtained  multiple  R  was  .839,  with  only 
five  Items  entered  Into  the  equation.  These  results  are  summarized  In  Table  II . 


Table  II 

Summary  of  Multiple  Correlation  Analysis 
Predicting  Total  RAG  Grade 


Variable 

Selected 

Cumulative 
Shrunken  R 

Power/Nose  Control 

FMLP/CQ 

.635 

Offensive  ACM 

PT 

.729 

Scan 

FMLP/CQ 

.783 

Baste  Alrwork 

PS 

.801 

VFR  G/S  Control 

PF 

.839 

DISCUSSION 


As  the  correlations  from  Table  I  Indicate,  the  PT  and  FMLP/CQ  stage  Items 
shared  the  most  variance  with  the  final  overall  RAG  grade.  This  Is  not  too  surprising— 
at  least  from  a  rational  viewpoint.  These  two  stages  are  perhaps  the  most  Important 
for  the  F-4  community.  Tootles  defines  the  mission  ob|eetlves  of  the  fighter  community, 
while  proficiency  in  carrier  landings  Is  essential  to  get  the  pilot  and  aircraft  safely  off 
and  onboard  the  ship.  Further  support  for  the  Importance  of  these  stages  Is  provided  by 
the  results  of  the  regression  analysis  predicting  the  final  overall  RAG  grade  from  the 
17  Items  selected.  Of  the  five  Items  entered  Into  the  equation,  those  from  the  PT  and 
FMLP/CQ  stages  accounted  for  61  percent  of  the  total  verla.^e  of  the  criterion,  while 
those  from  the  PS  and  PF  stages  accounted  for  only  an  additional  9  percent.  It  Is  olear 
from  these  findings  that  any  estimate  of  fleet  performance  mutt  be  heavily  weighted  by 
proficiency  In  Air  Combat  Maneuvering  and  Carrier  Landings. 

The  procedures  carried  out  In  this  Investigation  are  most  similar  te  an  Item 
analysis  In  tost  construction  theory.  In  both  Instances  the  goal  It  to  extract  from  the 
overall  criterion  that  portion  of  the  variance  which  ean  bo  reliably  eecountod  for  by 
scores  derived  from  a  small  subset  of  the  Individual  Items*  In  this  case  the  atm  was  to 
Isolate  the  most  Important  of  the  graded  skills  and  procedures  within  each  stage.  As 
expected,  only  a  few  of  the  graded  Items  were  found  to  be  highly  relevant  to  the  stage 
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grada  a  student  receives.  This  reflects  the  fact  that  certain  Items  simply  do  not 
discriminate  among  students.  In  most  Instances,  this  occurred  whenever  the  vast 
ma|or!ty  of  students  received  the  satne  rating  for  a  particular  Item, 

The  finding  that  five  of  the  Items  accounted  for  70  pereent  of  the  variance 
of  the  final  RAO  grade  Is  certainly  encouraging.  It  points  to  the  feasibility  of  obtain- 
Ing  valid  estimates  of  flight  ability  through  the  use  of  only  a  few  Item  measures.  The 
authors  feel  this  to  be  of  great  Importance  In  that  brevity  Is  one  of  the  key  requirements 
of  any  rating  form  to  be  completed  by  operational  squadron  personnel .  It  Is  hoped  that 
by  Isolating  the  most  Important  graded  skills  and  procedures  In  the  RAO,  a  valid 
measure  of  fleet  performance  con  be  constructed. 
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APPENDIX  A 


SUMMARY  OF  F-4  REPLACEMENT  PILOT  FLIGHT  SYLLABUS 


FAMILIARIZATION  (PF):  During  the  PF  stag#,  the  replacement  pilot  (RP)  learns  the 
baste  requirements  and  procedures  for  operating  the  F-4  aircraft.  These 
include  pre-flight  procedures,  starting,  taxiing,  take-offs,  and  landings. 

He  Is  also  Introduced  to  aircraft  maneuvering,  single  engine  performance, 
aerobatics,  baste  formation  flight,  and  night  operations. 

INSTRUMENTS  (PI):  During  the  PI  stage,  the  RP  Is  Introduced  to  Instrument  flight  In 
the  F-4  aircraft.  He  learns  Instrument  take-offs,  standard  departures, 
penetrations,  approaches,  and  landings.  Upon  completion  of  this  stage,  he 
Is  certified  to  be  Instrument  qualified  In  the  F-4. 

WEAPONS  SYSTEMS  (PS):  During  the  PS  stage,  the  RP  Is  Introduced  to  the  operations 
of  the  different  missile  systems  In  the  F-4.  Heavy  emphasis  ts  placed  upon 
the  development  of  skills  In  the  basic  intercept  technique. 

CONVENTIONAL  WEAPONS  (PW):  During  the  PW  stage,  the  RP  Is  Introduced  to  con¬ 
ventional  air-to-ground  weapons  systems.  He  Is  also  Introduced  to  basic  F-4 
tactics,  air  refueling,  and  navigational  procedures. 

TACTiCS  (PT):  During  the  PT  stage,  the  RP  Is  Introduced  to  air  combat  maneuvering 
procedures  In  the  F-4.  He  Is  Introduced  to  those  enemy  weapons  systems 
which  he  Is  likely  to  encounter  and  learns  the  best  tactical  procedures  to 
defeat  these  systems. 

FIELD  MIRROR  LANDING  PRACTICE/CARRIER  QUALIFICATION  (FMLP/CQ):  During 
this  stage ,  the  RP  Is  prepared  for  carrier  landings  under  both  day  and 
night  operations. 
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